The law of averages

Definition 8.3 We say that the sequence of random variables converges in mean square to the (limit) random variable if If this holds, we write ‘ in mean square as ’.
Theorem 8.6 (Mean-square law of large numbers) Let be a sequence of independent random variables, each with mean and variance . The average of the first n of the satisfies, as ,

Chebyshev’s inequality and the weak law

Definition 8.12 We say that the sequence of random variables converges in probability to as if If this holds, we write ‘ in probability as ’.
It turns out that convergence in mean square is a more powerful property than convergence in probability.
Theorem 8.14 If is a sequence of random variables and in mean square as , then in probability also.
Theorem 8.15 (Chebyshev’s inequality) If is a random variable and , then
Theorem 8.17 (Weak law of large numbers) Let be a sequence of independent random variables, each with mean and variance . The average of the first of the satisfies, as ,

Theorem 8.25 (Central limit theorem) Let , . . . be independent and identically distributed random variables, each with mean and non-zero variance . The standardized version of the sum satisfies, as , it may be written as where is a random variable with this standard normal distribution.
Theorem 8.27 (Continuity theorem) Let be a sequence of random variables with moment generating functions and suppose that, as , Then,

Let , and define the so-called Fenchel–Legendre transform of by
Theorem 8.36 (Large deviation theorem) Let be independent, identically distributed random variables with mean 0, whose common moment generating function is finite in some neighborhood of the origin. Let be such that . Then and That is to say, decays to 0 in the manner of

Definition 8.45 The sequence is said to converge in distribution, or to converge weakly, to as if where is the set of reals at which the distribution function is continuous. If this holds, we write .
Theorem 8.46 If is a sequence of random variables and in probability as , then .
Theorem 8.47 Let be a sequence of random variables which converges in distribution to the constant . Then converges to in probability also.
Theorem 8.50 (Continuity theorem) Let be random variables with characteristic functions Then if and only if